On Continuous Time Markov Games with Countable State Space

This paper is a continuation of our papers [61 and [71 and is concerned with a continuous time Markov game in which the state space is countable and the action spaces of player I and player n are compact metric spaces. In the game, the players continuously observe the state of the system and then choose actions. As a result, the reward is paid to player I from player n and the system moves to a new state by the known transition rates. Then we consider the optimization problem to maximize the total expected discounted gain for player I and, at the same time, to minimize the total expected loss for player n as the game proceeds to the infinite future. We show that such a two-person zero-sum game is strictly determined and both players have optimal stationary strategies.